Hannes Datta
Using virtual environment '/Users/hannesdatta/.virtualenvs/r-reticulate' ...
Using virtual environment '/Users/hannesdatta/.virtualenvs/r-reticulate' ...
We're about to start with the first lecture of this class.
If you haven't done so, please
Substantive interests
Methodological interests
With web scraping, you can capture anything you can view in a web browser
With APIs, you obtain official data from a firm in a programmatic way
Let's first import some packages
And then call a particular URL (check it out in your browser!)
Finally, let's retrieve the weekly top 15 songs (we use HTML tags and attribute-value pairs for this)
Works with any website, even anything you see in a browser (e.g., apps)
APIs are official interfaces by firms for programmers to extract or submit data, or obtain access to an algorithm
They work like websites (i.e., you can call them with the same snippets as before), but usually you need to pay or at least sign up for the service
let's structure the output in the JSON format
What’s the last app/website that made you say ‘wow’?
If you could only use three apps/websites for the next year, which ones would you pick and why?
What's a niche online community you're part of that most people don’t know about?
What’s the last thing you saw on TikTok that made you stop scrolling?
Imagine an AI tool that could make you internet-famous overnight. What would it do?
Let's talk about it right now…
Why do we need a course on this? Isn't this how research is always done?
Yes, but collecting web data is different from other datasets!
Each project is totally unique - that's why there is no universal “best way” to approach things…
Weekly modules, structured along the methodological framework
Project in which you put into practice your skills (40% of your grade)
On-campus computer exam (60% of your grade)
Specifics
Evaluation
Visit https://odcm.hannesdatta.com!
Course website is your #1 resource, Canvas used for
Do all students have access to Canvas?
→ quick feedback loops in first few weeks
When you run into trouble, this is your way out!

Investment in research skills
Essential skills for entrepreneurs
Showcast expertise in coding